At least as far back as Sir Charles Wheatstone's invention of the stereoscope in the 1840s, people have been making stereoscopic 3-D images using various methods in an effort to capture and play back the most realistic reproduction of past experiences and entertaining presentations. Ever since the invention and development of television, beginning in the 1930s and continuing to the present, people have tried to produce a practical 3-D television system (especially for viewing by multiple viewers simultaneously) utilizing various methods that were forerunners of today's most significant competing 3-D TV systems. With the advent of high-brightness, computer-controlled digital projectors (first developed in the 1980s by the present inventor) that can project polarized right-eye and left-eye stereoscopic images without any need of operator alignment, 3-D movies in theaters have finally become mainstream and profitable. Their widespread acceptance, success, and rapid growth have created a new high level of consumer acceptance of, and demand for, stereoscopic entertainment utilizing 3-D glasses. This has resulted in an unprecedented demand for a practical, high quality, affordable stereoscopic TV system solution. The predominant competing 3-D TV display approaches center around four different viewing technologies: Anaglyphic (or other colored) Glasses, Polarized Glasses, Shutter Glasses, and Glasses-Free Viewing. Several proposals have also been made as to how to send two video streams over a single video channel since stereoscopic 3-D requires separate images for each eye of the viewers.
Anaglyphic Glasses Viewing
In 1918, Christian Berger filed a patent application (now U.S. Pat. No. 1,295,842) for a system to separate the visibility of two images by making them of complementary colors (such as green and red) and using complementary filters to restrict viewing by each eye to one or the other of the images. In 1920, he filed an additional patent application (now U.S. Pat. No. 1,422,527) utilizing the same technique to display anaglyphic stereoscopic 3-D images (the anaglyph technique was first proposed by the painter J. C. d'Almeida in 1858). In 1932, Frank Crosier filed an application (now U.S. Pat. No. 2,056,600) for a camera that shot anaglyphic stereoscopic movies wherein one eye's image was shot in red while the other eye's image was shot in blue and green(cyan). 3-D could be seen by spectators wearing red and blue/green (cyan) glasses. In 1941, John Baird filed for a patent (now U.S. Pat. No. 2,349,071) for an anaglyphic stereoscopic television system utilizing spinning anaglyphic color-wheel filter glasses for the viewers. Later that year, Alfred Goldsmith of RCA applied for a patent (now U.S. Pat. No. 2,384,260) for a full color version of the Baird system utilizing spinning multicolor-wheel filter glasses for the viewers. In 1977, Alvin and Mortimer Marks filed for a patent (now U.S. Pat. No. 4,134,644) to improve the color fidelity of anaglyphic stereoscopic images by using green and magenta filters.
Polarized Glasses Viewing
In 1938, Chalon Carnahan of the Sylvania Corp. filed for a patent (now U.S. Pat. No. 2,301,254) for a stereoscopic television system wherein the right and left images were interlaced, alternate lines on the display screen were polarized perpendicularly, and the viewers wore polarized glasses. In 1945, Ray Kell of RCA applied for a patent (now U.S. Pat. No. 2,508,920) for a stereoscopic color TV system which alternately transmitted the left and right images in sequence as alternate fields, utilizing a rotating polarizing filter in front of the display and polarized glasses for the viewers. In 1963, the current inventor developed a stereoscopic TV system utilizing two displays placed at right angles to each other, which were also polarized perpendicular to each other, whose images were overlapped with a beam combiner for 3-D viewing with polarized glasses. Similar systems are available today utilizing LCDs, for instance, from Planar Corp. In 1984, the current inventor built the world's first LCD projector, filing a patent on it in 1987 (application number 140,233, which became U.S. Pat. No. 5,012,274 after refiling in 1988), in which he disclosed methods of projecting stereoscopic 3-D (utilizing either front-projection or rear-projection) with an LCD projector, wherein the viewers wear polarized glasses or view the 3-D without glasses through a special lenticular screen. This technology was expanded to include any type of light valve projector in his 1991 patent application (now U.S. Pat. No. 5,300,942).
In 1990, Sadeg Faris applied for a patent (now U.S. Pat. No. 5,327,285) disclosing how to make a micro-polarizer sheet to be placed over an LCD screen, polarizing certain pixels in one direction and other pixels in the perpendicular direction. Also in 1990, he applied for a patent (now U.S. Pat. No. 5,537,144), which disclosed how to use the micro-polarizer attached to an LCD to provide a stereoscopic video display for viewers wearing polarized glasses. In 2006, Arisawa Corp., partly under a license from him (in addition to their own technological improvements), began making and selling other companies' LCD displays with attached micro-polarizer filters and the electronics to take two simultaneous stereoscopic video signals, provided in any one of four formats, and interlace them so that the even lines display one eye's image and the odd lines display the other eye's image, allowing viewers to see a stereoscopic 3-D image when wearing polarized glasses. The four formats are: 1. One image above the other image within a single frame, each image being compressed to half its height, 2. One image beside the other within a single frame, each image being compressed to half its width, 3. Sequentially presented full frames, first a left-eye frame, then a right-eye frame, and then a left-eye frame, and so on, and 4. A sequence of frames in which each frame consists of a left-eye image interlaced with a corresponding right-eye image.
In 1993, Faris applied for a patent (now U.S. Pat. No. 6,111,598) using his micro-polarizer technology, but using a more complex cholesteric micro-filter array structure (disclosed by him in 1991 in U.S. Pat. No. 5,221,982), using a direct-view LCD display and polarized color multiplexing, to improve upon an earlier technology invented by Karen Jachimowicz and Ronald Gold, filed in 1989 (now U.S. Pat. No. 4,995,718), that also utilized polarized color multiplexing, but with a single full-color video projector, with the addition of an electronically controlled liquid crystal polarization rotator (such as a pi-cell), which provided viewing of projected stereoscopic 3-D images by viewers wearing polarized glasses. A pi-cell is a liquid crystal polarization rotator that was developed at Tektronix Corp. in 1984. It is capable of rotating the polarization of light 90 degrees, quickly and on demand. The pi-cell is used today in front of the projection lens of digital projectors in most movie theaters showing 3-D movies. In his patent, Faris also described methods using polarized glasses with a pi-cell polarization rotator placed over a CRT or other display screen, and a system utilizing complex image processing to form spectrally multiplexed images which are viewed through complex active polarizing glasses that use six different wavelength bands for each eye.
Shutter Glasses Viewing
In 1922, Adelbert Ames Jr. applied for a patent (now U.S. Pat. No. 1,673,793) utilizing shutter glasses to view 3-D images. In 1941, Ramon Oriol filed for a patent (now U.S. Pat. No. 2,365,212) for making motion pictures in which successive frames of the movie were alternately right eye and left eye views, with the spectators wearing electro-mechanical shutter glasses. In 1969, Karl Hope applied for a patent (now U.S. Pat. No. 3,621,127) for stereoscopic TV and movie systems which alternately displayed left and right images while the viewer wore wireless shutter glasses whose shutters opened and closed by mechanical action. The invention of the liquid crystal display (LCD) in the 1970s, which led to many important breakthroughs in the development of stereoscopic TV displays, also led to improved shutter glasses for 3-D viewing. In 1972, Shunsei Kratomi applied for a patent (now U.S. Pat. No. 3,737,567) for a stereoscopic, alternating frame system using liquid crystal shutter glasses. In 1974, a similar system was described in a patent application (now U.S. Pat. No. 3,821,466) by John Roese, who later improved upon it in his 1980 patent application (now U.S. Pat. No. 4,424,529) by remotely triggering the shutter glasses, making them wireless. In 1985, Lenny Lipton, Michael Starks, James Stewart, and Lawrence Meyer filed an application for a patent (now U.S. Pat. No. 4,523,226) disclosing an improved shutter-glasses-based stereoscopic display system, wherein the display operated at 120 Hz, rather than the usual 60 Hz, to diminish the flicker seen in all previous stereoscopic shutter glasses systems.
In 1982, Larry Hornbeck of Texas Instruments applied for a patent (now U.S. Pat. No. 4,441,791) for a new type of display device utilizing an array of micro-mirrors on a deformable membrane, referred to as a deformable mirror device (DMD), eventually also being referred to as a digital light processor (DLP). This light-valve based system is capable of very fast frame rates. In 2004, Stephen Marshall, Michael Allbright, and Bill McDonald, also of Texas Instruments, applied for a patent (U.S. pending application publication number 20050146540) that added the technique of image dithering to the DMD to increase the apparent resolution of the image. This was done by doubling the display frequency to 120 Hz and displaying the same image data twice in two different positions (½ a pixel apart), making the boundaries of the individual pixels less noticeable. In 2006, Keith Elliott, David Hutchison, Henry Neal, and Bradley Walker, also of Texas Instruments, applied for a patent (U.S. pending application publication number 20080036854) to use the dithered DLP for stereoscopic 3-D projection, displaying the left and right eye images sequentially, one after another. Utilizing this system with liquid crystal shutter glasses reduces perceived flicker since the DLP projector operates at 120 Hz (rather than conventional television's 60 Hz), allowing each eye to see 60 flashes per second. To further minimize the visibility of pixels and flicker, the left and right images are displayed as alternating checkerboard patterns (with the checkerboard squares rotated to form diamond shapes). With this method, one pattern (sent to one of the viewers' eyes through shutter glasses), beginning in the first row of pixels with a pixel showing image information, is followed by a black pixel, which is followed by another pixel showing image information, and so on, while the second pattern, sent to the other of the viewers' eyes, begins with a black pixel, followed by a pixel showing image information, followed by a black pixel, and so on. Every other field is shifted horizontally by the width of half a pixel, with respect to the previous field, to provide the appearance of higher resolution and to reduce the appearance of pixels. In 2007, this system, built into a cabinet utilizing rear-projection, was licensed to Samsung and Mitsubishi, which are currently marketing it as a 3-D-ready television.
Glasses-Free Viewing
In 1930, Aloysius Cawley filed for a patent (now U.S. Pat. No. 2,118,160) for a color stereoscopic system projecting polarized images onto a prismatic screen for 3-D viewing without any need for special glasses. In 1950, Arthur Wright filed for a patent (now U.S. Pat. No. 2,621,247) for a color stereoscopic TV system utilizing the color wheel and a lenticular type of screen for glasses-free 3-D viewing. In 1957, Albert Abramson filed for a patent (now U.S. Pat. No. 2,931,855) disclosing designs for stereoscopic television utilizing either one of a lenticular lens or a barrier screen on the front of the display for glasses-free 3-D viewing. Although many people have also proposed and patented various lenticular-based auto-stereoscopic television displays throughout the years, the first TV system to get rid of the obvious and annoying black lines visible between columns of pixels on an LCD screen utilizing a lenticular lens was Cornelis van Berkel of the Philips Corp., who filed a patent application on it (now U.S. Pat. No. 6,064,424) in 1997, wherein he tilted the lenticular lenses with respect to the columns of pixels. The system displays nine different perspective views over a few predefined angles, within which viewers can see an auto-stereoscopic image with parallax, without the need to wear any special glasses. Through the years, many people have also proposed and patented auto-stereoscopic LCD television displays utilizing a parallax barrier. One example is disclosed in a patent application filed in 1995 (now U.S. Pat. No. 5,640,273) by Goro Hamagishi of the Sanyo Corp. In recent years, NewSight Corp. has been selling parallax barrier LCD stereoscopic television displays in which the parallax barriers are also tilted with respect to the columns of pixels on the LCD, providing a similar auto-stereoscopic viewing experience (to the lenticular version), also without the need to wear any special glasses.
Drawbacks of the Various 3-D TV Systems Listed Above
This history (above) summarizes the advances made in the technologies that have proven to be the most promising in producing working 3-D television displays. However, none of them have succeeded in overcoming all obstacles regarding required quality, simplicity, practicality, and affordability to enable any of them to be accepted by the marketplace and become the standard for consumer stereoscopic 3-D television.
Anaglyphic (and other color-separation) glasses-viewed displays, still in use today, fail to provide natural color imagery and usually produce eyestrain and headaches after a relatively short period of time. This is due to color rivalry and brightness imbalance between the images supplied to the two eyes of the viewers. In addition, the difficulty in providing filters that match the colors emitted by TV sets and efficiently separate the images by their color, due to the overlapping spectra of TV phosphors or filters, causes ghosting that reduces the 3-D effect, blurs image elements in front of, and behind, the image plane, and provides an annoying experience.
Spinning filter wheels and other mechanical devices are unreliable, prone to breakage, noisy, and bulky.
Perpendicular dual displays are expensive, bulky, and ugly.
Lenticular displays provide a limited angle of view, after which an annoying pseudoscopic image, followed by an annoying jump, can be seen. In addition, the full-screen image resolution is divided by the number of images being shown at different angles, dramatically reducing the resolution of each image. The display also only works within a narrow range of distances from the display. Being too close or too far from the display produces eyestrain and eliminates the appearance of 3-D. In addition, due to image crosstalk, two or more images can always be seen at the same time, causing ghosting and blurring of image elements appearing too far in front of, or behind, the image plane. This limits the amount of depth that can be shown with this technology.
Parallax barrier displays have the same drawbacks as lenticular displays, with the added drawback that they produce a dimmer image.
The polarized color multiplexing projector system technology requires an electronic polarization rotator in front of a projector, making direct view stereoscopic display with conventional direct-view televisions impossible.
The combination of polarized color multiplexing with micro-polarizer technology utilizes a complex system requiring special cameras and either an expensive complex series of large (as big as the entire display) electro-optical (pi-cell) panels, or, alternately, an expensive, complex, never-before-tried cholesteric micro-polarizer filter screen installed in front of a display. Color multiplexing active cholesteric glasses requires 6 wave-bands and complex processing.
Shortcomings of Today's Leading 3-D TV System Candidates
As of the filing date of this application there is no compression and transmission system capable of providing the ability to encode stereoscopic image pairs over a single TV channel wherein each image of the stereo pair is transmitted with full high-definition resolution while enough unique (representing different points in time) stereo pairs are transmitted per second to provide the full standard video frame rate for each eye. Nor is there a way to display stereoscopic image pairs on readily available TV monitors wherein each image of the stereo pair is displayed with full high-definition resolution while enough unique (representing different points in time) stereo pairs are displayed per second to provide the full standard video frame rate for each eye. This is primarily because the current HDTV standard was designed to transmit and display 30 frames per second, with each frame having a resolution of 1920 by 1080 pixels. Doubling the information content (required to send and display stereo pairs of images) would produce data that would not fit into a single TV channel utilizing prior art technologies and the doubled information content could not be displayed on conventional TV monitors utilizing prior art technologies since they only have 1920×1080 pixels. In other words, the current HDTV standard does not support the amount of information required to transmit and display two images needed for 3-D at the maximum resolution and frame rate for both required stereoscopic images. Sensio has developed an algorithm which is currently used most often in 3-D encryption and decryption, that could be used for broadcast application. However, they also discard half of the pixels from each eye's image to cut the bandwidth requirement of each image of each stereo pair in half to allow transmission on a single channel.
Currently there are mainly only two types of stereoscopic 3-D TV systems being sold in “commercial quantities.” Today's front-runners for an acceptable consumer 3-D TV system (referred to as “3-D-ready TVs”) are: 1. The DLP-based rear projection, LED-illuminated LCD, and plasma-panel-based televisions viewed with shutter glasses (the “alternate frame” system); and 2. The simple micro-polarizer LCDs viewed with polarized glasses (the “alternate line” system).
The first type of system, the “alternate frame” system, alternately drops every other frame from each of the two streams of frames intended for the viewer's two eyes and transmits the remaining frames, one at a time, in an interleaved sequence. One version of this system, utilizing a digital micro-mirror device (DMD) developed and patented by Texas Instruments, has mainly been productized in rear projection TV sets sold primarily by Samsung and Mitsubishi. Another version of this “alternate frame” system, sold by Samsung, Sony, L G, and Panasonic, utilizes either a flat-panel plasma or LED-illuminated LCD display. Both “alternate frame” systems utilize shutter glasses to direct each image of each stereo pair to the proper eye of the TV viewers. The shutter glasses are not inexpensive (currently being sold for $150 each) and produce the appearance of some noticeable flicker (especially in peripheral vision), since each eye alternately switches from total black to total image brightness, giving the flicker the highest possible contrast and visibility. During one second, each eye of the viewer is shown image information derived from only 15 unique frames. Each eye's image is “jumpy” since every other original frame of action for each eye is missing. To reduce the appearance of flicker, the frames are stored in buffers on the receiving end and either each frame is flashed at least twice or interpolation based on motion prediction algorithms is used to produce intermediate frames. However, even with monitors operating at 120 Hz, the rods in many people's eyes (which make up 125 million of the 130 million receptors of each eye and which are found predominantly in the periphery of the retinas) can still detect the flicker, which is annoying and can create eyestrain and headaches after a period of time.
Since each eye sees black at least 50% of the time, the shutter cuts the overall perceived TV image brightness to half (or less). In addition, since the glasses use polarizers, while light coming from the TVs are not polarized, perceived brightness is further reduced by an additional 60 to 70%, bringing the visible image brightness down to about 30-35%. The TVs initially required the hook up and use of an external computer, with installation of special software, reducing their potential audience (although this computer function is now built into 3-D-ready sets). To synchronize the TV picture with the glasses, an infrared transmission system is used which is directionally sensitive, causing loss of synchronization (and 3-D), creating a brief double image if the viewer's head is not in the right position and/or orientation.
The other type of stereoscopic 3-D TV system on the market, the “alternate line” system, alternately drops every other line from each frame in the two streams of stereo pairs intended for the viewer's two eyes, interlacing the remaining lines from each stereo pair of images into a single frame. When each of these interlaced frames is displayed on the TV monitor, the odd lines display one eye's image while the even lines display the other eye's image. Currently, while this system is becoming popular in Europe, the US has very limited sales. Since less of these sets have been sold, their prices are much higher than alternate-frame (shutter glasses) 3-D TVs. Even though the alternate-line system produces less ghosting than the alternate-frame system, the increased profit margin on sales of shutter glasses has resulted in this imbalance in sales volumes of the two systems.
This alternate line system utilizes micro-polarizers, each the height of one scanning line, adhered to the front of the monitor so that each scanning line is polarized perpendicular to the next scanning line. Wearing simple, inexpensive polarized glasses to direct each image of each stereo pair to the proper eye of a TV viewer, one of the viewer's eyes will see only the odd lines while the other of the viewer's eyes will see only the even lines, providing a 3-D image to the viewer. When viewing this image with both eyes and polarized glasses, however, each eye sees an image wherein every other line appears black. This cuts the resolution and brightness of each eye's image in half. It also introduces jaggies, creating image artifacts, and eliminates fine detail, often making small, originally readable text unreadable.
This is illustrated in FIGS. 1a-1h. FIG. 1a is a high-definition image. FIG. 1b is a close-up view of part of the image showing extremely fine text which is parallel to the horizontal. FIG. 1c is a close-up view of another area on the image showing some small readable text situated at nearly a 45° angle. FIG. 1d shows the actual display pixels in a portion of the image in FIG. 1c, as seen through a magnifying lens. FIG. 1e depicts the same image shown in FIG. 1a except that every other line is black. This is what one eye would see with this “alternate line” system described above. FIG. if shows a close-up view of the same area shown in FIG. 1b, but with every other line being black. With every other line of data missing, the fine text is noticeably difficult to read. FIG. 1g shows a close-up view of the small previously readable text situated at nearly a 45° angle depicted in FIG. 1c, again with every other line being black. This text is now virtually unreadable. FIG. 1h shows the actual display pixels of the display as in FIG. 1d, as seen through a magnifying lens, with every other line being black. Notice also that straight lines in FIG. 1a (such as the lines depicting the edge of the door and the bottom edges of the front bumper and the left side of the car body) become jagged in FIG. 1d. 
Another drawback of the alternate line system is that the micro-polarizer is attached to the outside surface of the LCD, thereby being spaced a distance away (the thickness of the LCD glass) from the actual pixels. Consequently, there is a limited vertical viewing angle (you can't stand up or lie on the floor when viewing this display), beyond which parallax error (between pixels and micro-polarizers) causes both images to be seen by each eye, eliminating the 3-D and creating an annoying double image.
As stated above, since the micro-polarizers are mounted on the outside of the glass surface of the monitor, while the liquid crystals which form the image are on the opposite side of the glass, a parallax error is created, producing a limited vertical angle of view. This is depicted in FIG. 2. The LCD monitor 210 comprises liquid crystal material 216 suspended between two glass plates, one plate 212 on the viewer side of the LCD and one plate 214 on the light source side (light source not shown). Odd horizontal scanning lines, such as those indicated at 220, are viewed through micro-polarizer stripes 270 and through one polarizer 244 in the viewer's polarized glasses 240. Each odd scanning line 220 subtends an angle 250 through each micro-polarizer stripe 270. The even scanning lines 230 are similarly visible through corresponding micro-polarizer stripes 280 subtending a similar angle, and are visible through the other polarizer 242 in the viewer's glasses 240. The viewer sees 3-D through the polarized glasses 240 when viewing the LCD screen 210 within the subtended viewing cone 250. However, if the viewer attempts to view the screen from an angle 260 outside of the viewing cone 250 (such as happens when standing up), each eye will see a double image and no 3-D. At such an angle, light from odd scanning lines 220 can be partially seen through micro-polarizer stripes 270 and through polarizer 244 in the viewing glasses 240 as well as through micro-polarizer stripes 280 and through polarizer 242 in the viewing glasses 240, simultaneously. The same is true for even scanning lines 230. Currently this type of system is being sold by Pavonine, Zalman, Hyundai, and JVC.
All systems suffer from the fact that, to display a stereoscopic image, two images must simultaneously be transmitted and displayed over a single television channel, which is only large enough for one image. Consequently, displaying the two images to create stereoscopic viewing reduces the overall information content (resolution or motion smoothness of each eye's image) by 50% (although time-multiplexing and image shifting techniques used in the DLP-based system reduce the ability to notice this loss somewhat).
No 3-D System Available for Existing Displays
The switch to 3-D consumer television would be most successful if already-installed home televisions, computers, hand-held devices, and projectors, as well as newly manufactured made-for-3-D TV displays, could be utilized to display 3-D. Although new types of displays (LCD, plasma, DLP, LED, OLED, Laser, etc.) are becoming very popular, and CRT televisions are no longer being sold, there is still a very large established base of CRT television ownership throughout the world, and CRT televisions have the longest lifetime (often 20 years or more). Consequently, the best 3-D TV solution would also allow owners of any type of television, including current CRT set owners, to watch 3-D TV. Unfortunately, however, none of the current 3-D television technologies work with existing conventional 2-D televisions. Additionally, since many “3-D-ready” TV displays (utilizing the 3-D display technologies detailed above) have already been sold to consumers, any new 3-D TV technology would be more widely accepted if it was also compatible with such existing 3-D-ready displays.
The Chicken/Egg Barrier to Development of a 3-D Solution
Another problem that has to be overcome before successful consumer 3-D television can become a reality is the chicken/egg problem created by the lack of 3-D content and the lack of an installed base of consumers with 3-D-capable TV sets. Without an established “best solution” to form the basis for a 3-D TV standard, most TV manufacturers can't justify the expenditures necessary to develop, mass produce, mass-market, and aggressively sell a 3-D TV system to consumers. Without confirmation that there will be a widespread installed base of 3-D TV set owners, and without the knowledge of what the 3-D television standard will be, content providers can't justify the expense of large-scale production of 3-D television content. Thus, in addition to the need for a practical 3-D TV technology to be found, a source of virtually unlimited 3-D content must also be developed for the transition to 3-D TV to occur.
To foster the development of a 3-D TV industry, there is a strong need for a 3-D television technology that can work with all existing TV sets of any kind (without modification), as well as with all new TV sets that will be manufactured and sold in the foreseeable future. These sets need to provide at least 30 frames (taken at different times) per second (for NTSC) or at least 25 frames (taken at different times) per second (for PAL) for each eye, the highest possible resolution displayed with top-quality 3-D (showing bright and sharp images at all depths, with no ghosting, full undistorted color, viewability from all angles, and no perceivable flicker, eyestrain, or headaches) and need to be able to utilize currently available single-channel bandwidths, at an affordable price.